Fatman: Cost-saving and reliable archival storage based on volunteer resources

نویسندگان

  • An Qin
  • Dianming Hu
  • Jun Liu
  • Wenjun Yang
  • Dai Tan
چکیده

We present Fatman, an enterprise-scale archival storage based on volunteer contribution resources from underutilized web servers, usually deployed on thousands of nodes with spare storage capacity. Fatman is specifically designed for enhancing the utilization of existing storage resources and cutting down the hardware purchase cost. Two major concerned issues of the system design are maximizing the resource utilization of volunteer nodes without violating Service Level Objectives (SLOs) and minimizing the cost without reducing the availability of archival system. Fatman has been widely deployed on tens of thousands of server nodes across several datacenters, provided more than 100PB storage capacity and served dozens of internal massdata applications. The system realizes an efficient storage quota consolidation by strong isolation and budget limitation, to maximally support resources contribution without any degradation on host-level SLOs. It firstly improves data reliability by applying disk failure prediction to minish failure recovery cost, named fault-aware data management, dramatically reduces the MTTR by 76.3% and decreases file crash ratio by 35% on real-life product workload.

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Using Storage Class Memory for Archives with DAWN, a Durable Array of Wimpy Nodes

The long life and low usage of archival data make cost considerations paramount. Today, most archival storage architectures depend on magnetic or optical media such as tape and disk because they have a low initial cost per byte. The high initial cost of storage class memories (SCMs) has been seen as prohibitive for archival use. Nevertheless, SCMs have many advantages for archival use, includin...

متن کامل

Evaluation of Peak Shifting and Energy Saving Potential of Ice Storage Based Air Conditioning Systems in Iran

Thermal energy storage (TES) system has been introduced as a practical facility for shifting load from peak hours to off-peak hours. Because of different energy consumption during day and night, peak and off peak period is created on load curve. Ice storage technology which is a kind of TES system, is implemented in different points of the world with the purpose of solving load shifting problem...

متن کامل

Protection of Archival Documents from Photochemical Eects

Purpose: ­The purpose of this paper is to highlight the destructive effects of light on archival documents/paper materials. ­The research aims to explain the mechanism of photochemical degradation and the damaging effect of light on paper. It also tells us about the measures to be adopted to control the deteriorating effects of light on paper step by step. Design/Methodology/Approach: Th­e res...

متن کامل

Configuring and Tuning Archival Storage Systems

Archival storage systems must operate under stringent requirements, providing 100% availability while guaranteeing that data will not be lost. In this paper we explore the multiple interconnected subsystems that must be tuned to simultaneously provide high data transfer rates, high transaction rates, and guaranteed meta-data backup. We examine how resources must be allocated to the subsystems t...

متن کامل

Optimisation techniques for data distribution in Volunteer Computing

Volunteer Computing is a new paradigm of distributed computing where the ordinary computer owners volunteer their computing power and storage capability to scientific projects. The increasing number of internet connected PCs allows Volunteer Computing to provide more computing power and storage capacity than what can be achieved with supercomputers, clusters and grids. However, volunteer comput...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

عنوان ژورنال:
  • PVLDB

دوره 7  شماره 

صفحات  -

تاریخ انتشار 2014